8 research outputs found
Understanding (Un)Written Contracts of NVMe ZNS Devices with zns-tools
Operational and performance characteristics of flash SSDs have long been
associated with a set of Unwritten Contracts due to their hidden, complex
internals and lack of control from the host software stack. These unwritten
contracts govern how data should be stored, accessed, and garbage collected.
The emergence of Zoned Namespace (ZNS) flash devices with their open and
standardized interface allows us to write these unwritten contracts for the
storage stack. However, even with a standardized storage-host interface, due to
the lack of appropriate end-to-end operational data collection tools, the
quantification and reasoning of such contracts remain a challenge. In this
paper, we propose zns.tools, an open-source framework for end-to-end event and
metadata collection, analysis, and visualization for the ZNS SSDs contract
analysis. We showcase how zns.tools can be used to understand how the
combination of RocksDB with the F2FS file system interacts with the underlying
storage. Our tools are available openly at
\url{https://github.com/stonet-research/zns-tools}
Understanding (Un)Written Contracts of NVMe ZNS Devices with zns-tools
Operational and performance characteristics of flash SSDs have long been associated with a set of Unwritten Contracts due to their hidden, complex internals and lack of control from the host software stack. These unwritten contracts govern how data should be stored, accessed, and garbage collected. The emergence of Zoned Namespace (ZNS) flash devices with their open and standardized interface allows us to write these unwritten contracts for the storage stack. However, even with a standardized storage-host interface, due to the lack of appropriate end-to-end operational data collection tools, the quantification and reasoning of such contracts remain a challenge. In this paper, we propose zns.tools, an open-source framework for end-to-end event and metadata collection, analysis, and visualization for the ZNS SSDs contract analysis. We showcase how zns.tools can be used to understand how the combination of RocksDB with the F2FS file system interacts with the underlying storage. Our tools are available openly at \url{https://github.com/stonet-research/zns-tools}
A Survey on the Integration of NAND Flash Storage in the Design of File Systems and the Host Storage Software Stack
With the ever-increasing amount of data generate in the world, estimated to
reach over 200 Zettabytes by 2025, pressure on efficient data storage systems
is intensifying. The shift from HDD to flash-based SSD provides one of the most
fundamental shifts in storage technology, increasing performance capabilities
significantly. However, flash storage comes with different characteristics than
prior HDD storage technology. Therefore, storage software was unsuitable for
leveraging the capabilities of flash storage. As a result, a plethora of
storage applications have been design to better integrate with flash storage
and align with flash characteristics.
In this literature study we evaluate the effect the introduction of flash
storage has had on the design of file systems, which providing one of the most
essential mechanisms for managing persistent storage. We analyze the mechanisms
for effectively managing flash storage, managing overheads of introduced design
requirements, and leverage the capabilities of flash storage. Numerous methods
have been adopted in file systems, however prominently revolve around similar
design decisions, adhering to the flash hardware constrains, and limiting
software intervention. Future design of storage software remains prominent with
the constant growth in flash-based storage devices and interfaces, providing an
increasing possibility to enhance flash integration in the host storage
software stack
A Survey on the Integration of NAND Flash Storage in the Design of File Systems and the Host Storage Software Stack
With the ever-increasing amount of data generate in the world, estimated to reach over 200 Zettabytes by 2025, pressure on efficient data storage systems is intensifying. The shift from HDD to flash-based SSD provides one of the most fundamental shifts in storage technology, increasing performance capabilities significantly. However, flash storage comes with different characteristics than prior HDD storage technology. Therefore, storage software was unsuitable for leveraging the capabilities of flash storage. As a result, a plethora of storage applications have been design to better integrate with flash storage and align with flash characteristics. In this literature study we evaluate the effect the introduction of flash storage has had on the design of file systems, which providing one of the most essential mechanisms for managing persistent storage. We analyze the mechanisms for effectively managing flash storage, managing overheads of introduced design requirements, and leverage the capabilities of flash storage. Numerous methods have been adopted in file systems, however prominently revolve around similar design decisions, adhering to the flash hardware constrains, and limiting software intervention. Future design of storage software remains prominent with the constant growth in flash-based storage devices and interfaces, providing an increasing possibility to enhance flash integration in the host storage software stack
Performance Characterization of NVMe Flash Devices with Zoned Namespaces (ZNS)
The recent emergence of NVMe flash devices with Zoned Namespace support, ZNS
SSDs, represents a significant new advancement in flash storage. ZNS SSDs
introduce a new storage abstraction of append-only zones with a set of new I/O
(i.e., append) and management (zone state machine transition) commands. With
the new abstraction and commands, ZNS SSDs offer more control to the host
software stack than a non-zoned SSD for flash management, which is known to be
complex (because of garbage collection, scheduling, block allocation,
parallelism management, overprovisioning). ZNS SSDs are, consequently, gaining
adoption in a variety of applications (e.g., file systems, key-value stores,
and databases), particularly latency-sensitive big-data applications. Despite
this enthusiasm, there has yet to be a systematic characterization of ZNS SSD
performance with its zoned storage model abstractions and I/O operations. This
work addresses this crucial shortcoming. We report on the performance features
of a commercially available ZNS SSD (13 key observations), explain how these
features can be incorporated into publicly available state-of-the-art ZNS
emulators, and recommend guidelines for ZNS SSD application developers. All
artifacts (code and data sets) of this study are publicly available at
https://github.com/stonet-research/NVMeBenchmarks.Comment: Paper to appear in the https://clustercomp.org/2023/program
Performance Characterization of NVMe Flash Devices with Zoned Namespaces (ZNS)
The recent emergence of NVMe flash devices with Zoned Namespace support, ZNS SSDs, represents a significant new advancement in flash storage. ZNS SSDs introduce a new storage abstraction of append-only zones with a set of new I/O (i.e., append) and management (zone state machine transition) commands. With the new abstraction and commands, ZNS SSDs offer more control to the host software stack than a non-zoned SSD for flash management, which is known to be complex (because of garbage collection, scheduling, block allocation, parallelism management, overprovisioning). ZNS SSDs are, consequently, gaining adoption in a variety of applications (e.g., file systems, key-value stores, and databases), particularly latency-sensitive big-data applications. Despite this enthusiasm, there has yet to be a systematic characterization of ZNS SSD performance with its zoned storage model abstractions and I/O operations. This work addresses this crucial shortcoming. We report on the performance features of a commercially available ZNS SSD (13 key observations), explain how these features can be incorporated into publicly available state-of-the-art ZNS emulators, and recommend guidelines for ZNS SSD application developers. All artifacts (code and data sets) of this study are publicly available at https://github.com/stonet-research/NVMeBenchmarks
Key-Value Stores on Flash Storage Devices: A Survey
Key-value stores (KV) have become one of the main components of the modern storage and data processing system stack. With the increasing need for timely data analysis, performance becomes more and more critical. In the past, these stores were frequently optimised to run on HDD and DRAM devices. However, the last decade saw an increased interest in the use of flash devices because of their attractive properties. Flash is cheaper than DRAM and yet has a lower latency and higher throughput than HDDs. This literature survey aims to highlight the changes proposed in the last decade to optimise key-value stores for flash devices and predict what role these devices might play for key-value stores in the future